1. Field of the Invention
This invention relates to an operation picture displaying apparatus and method thereof for displaying a GUI (graphical user interface) picture which receives an operation to a computer in response to pointing by a pointing device such as a mouse with an improved operability.
2. Related Art
A GUI which displays an operation picture (icon) on the display screen of a monitor device, moves a pointer to a target icon to select it using a pointing device such as a mouse for allowing a user to input an operation correlated to the icon to a computer has been widely used.
For example, PUPA 5-73257 and PUPA 5-298023 (references 1, 2) disclose a GUI of improved operability by changing the color and shape of an icon depending on the moving speed of a cursor and selecting an icon depending on the direction of movement of a pointer and the distance from the icon.
However, with the method disclosed in the reference 1, a picture which is required for an operation is hidden by an icon which is enlarged. Even if only the color and the shape are changed without changing the size of the icon in this method, the difficulty involved when a precise manipulation such as precisely moving the pointer to a frame portion of the icon (window) is required is not different from a case where this method is not used.
The method disclosed in the reference 2 can not be applied to a case where it is desired to select a portion of an icon such as the frame of the icon (window).
A device to improve the operability of GUI includes displaying a function of an icon to which the mouse has moved in a picture called xe2x80x9ctool tipxe2x80x9d which looks like a balloon.
Because the tool tip is not displayed until the pointer moves to the icon, if the pointer is moved to a wrong icon despite the user wants to know the function of an aimed icon, the mouse operation required for moving the mouse is totally useless.
This invention is conceived in view of the above described problems of the prior art and aims at providing an operation picture displaying apparatus and a method thereof which makes the operation in doing a precise work such as pointing the frame of an icon (window) easy without sacrificing other operability to improve the operability of GUI.
It is also an object of this invention to provide an operation picture displaying apparatus and a method thereof which precisely predicts an icon (window) to be selected by a user in response to the operation of a pointing device to improve the operability of GUI.
It is another object of this invention to provide an operation picture displaying apparatus and a method thereof which precisely predicts an icon (window) to be selected by a user to improve the operability of GUI so as allow an operation to be effected to a part of an area of the icon (window) according to the prediction.
It is another object of this invention to provide an operation picture displaying apparatus and a method thereof which precisely predicts an icon (window) to be selected by a user to assist users operation by correctly displaying the function of the icon (window) according to the prediction.
First Picture Displaying Device
In order to achieve the above objectives, this invention provides a first operation picture displaying apparatus comprising: operation picture displaying means for displaying one or more operation pictures each of which is correlated to an operation to a computer in a display screen, pointer picture displaying means for displaying a pointer at a pointed position in said display screen in response to an external operation pointing an arbitrary position in said display screen, moving direction/speed detecting means for detecting the direction and the speed of movement of said pointer in said display screen, and operation picture selecting means for selecting either one of said operation pictures displayed in said display screen based on said detected moving direction and moving speed.
Preferably, said picture selecting means selects only said operation picture displayed in the direction of movement of said pointer when the moving speed of said pointer exceeds a predetermined threshold speed.
Preferably, said picture selecting means cancels the selection of said selected operation picture when the duration in which the moving speed of said pointer is below the predetermined threshold speed lasts longer than a predetermined threshold time.
Preferably, operation content displaying means is further provided to display the content of said operation correlated to said selected operation picture in said display screen.
Operation of the First Picture Displaying Device
The first picture displaying device of this invention displays an operation picture for GUI (GUI picture, icon/window and the like) correlated to a computer operation such as a file operation and starting a software, and a pointer indicating (pointing) the position in the display screen pointed by a pointing device such as a mouse in a display screen.
The first picture displaying device predicts and selects an icon to be selected by a user based on the speed and the direction of movement of the pointer and displays a picture (tool tip and the like) explaining a function correlated to the selected icon in association with the icon and the like
Operation Picture Displaying Apparatus
The operation picture displaying apparatus displays a GUI picture, i.e., a picture such as an icon and a window which is correlated to an operation to a computer in the display screen of a display device of the computer.
Pointer Displaying Means
The pointer displaying means displays a picture of a pointer such as a pointer of a mouse pointer at an arbitrary position on the display screen specified by an operation of a pointing device such as a mouse by a user.
Moving Direction/Speed Detecting Means
The moving direction/speed detecting means measures the display position of a pointer at a given time interval to detect to which direction and what distance (picture elements) the pointer moved (moving speed) in a given time.
Operation Picture Selecting Means
When the operation picture selecting means detects an event that the moving speed of the pointer exceeds a predetermined threshold speed, it determines that the user increases the moving speed of the pointer in an attempt to move the pointer to a target icon, predicts that an icon existing in the direction of the moment of the pointer (for example, within an angular range of +/xe2x88x92xcex8 with respect to the direction of movement of the pointer) is an icon for which the user targets and selects it.
When the picture selecting means detects an event that duration in which the moving speed of the pointer is below a predetermined threshold speed lasts longer than a predetermined threshold time when the picture selecting means is selecting an icon by prediction, the picture selecting means determines that the user no longer targets the icon selected by the prediction and cancels selection of the icon which has been so far selected by the prediction.
The operation picture displaying apparatus displays a function correlated to an icon selected by the prediction in the form of a tool tip, for example, in association with the pointer or the icon to assist the user to select the icon.
When the moving speed of the pointer decreases for a period longer than a given time and the selection by the prediction is cancelled, the operation picture displaying apparatus erases the display of the tool tip and the like
Second Picture Displaying Apparatus
The second operation picture displaying apparatus of this invention comprises; operation picture displaying means for displaying one or more operation pictures each of which is correlated to an operation to a computer in a display screen, pointer picture displaying means for displaying a pointer at a pointed position in said display screen in response to an external operation pointing an arbitrary position in said display screen, moving speed/distance detecting means for detecting the speed of movement of said pointer in said display screen and the distance (pointer distance) between said pointer and each of said operation pictures displayed in said display screen, and operation picture selecting means for selecting either one of said operation pictures displayed in said display screen based on said detected moving speed and said pointer distance.
Preferably, said operation picture selecting means selects one of said operation pictures which is within a predetermined distance range from said pointer only when said detected pointer distance is within said predetermined distance range and said detected moving speed is within a predetermined speed range.
Preferably, said operation picture selecting means cancels selection of said selected operation picture when said detected pointer distance is within said predetermined distance range and said detected moving speed exceeds an upper limit of said predetermined speed range or is blow a lower limit of said predetermined speed range for longer than a predetermined threshold time, or when said detected pointer distance is outside said predetermined distance range.
Preferably, pointer picture modifying means is further provided for modifying the picture of said displayed pointer in response to said operation correlated to said selected operation picture.
Preferably, operation input means is further provided for inputting said operation correlated to said selected operation picture to said computer in response to operation from the exterior.
Operation of the Second Operation Picture Displaying Apparatus
The second operation picture displaying apparatus of this invention displays an operation picture for GUI (GUI picture; icon/window and the like) which is correlated to a file operation and starting a software, and a pointer indicating (pointing) the position in the display screen pointed by a pointing device such as a mouse in the display screen.
The second operation picture displaying apparatus also predicts an operation which the user is effecting based on the moving speed of the pointer and the distance from the icon and changes the display of the pointer and the like depending on the prediction.
The second operation picture displaying apparatus further assists the user to select an icon, a window or a portion thereof (peripheral frame of the window) in response to operation (clicking a mouse button and the like) by the user after the display of the pointer has been changed.
Moving Speed/Distance Detecting Means
The moving speed/distance detecting means detects the moving speed v of the pointer in the manner similar to the moving direction/speed detecting means of the first operation picture displaying apparatus and detects how many picture elements (distance) the pointer is apart from the periphery of each icon.
Operation Picture Selecting Means
When the pointer is within a predetermined distance from a certain window and the moving speed v is within a predetermined speed range (0 less than v less than V; V is the upper limit of the speed range), the operation picture selecting means determines that the user is moving the pointer toward the window in an attempt to select it and lowering the moving speed to try to precisely place the pointer on the window, and predicts that the user is aiming at this window and selects it.
The operation picture selecting means may select by a prediction only a component of the window such as the frame thereof and the like which the user drags when the user changes the size of the window rather than necessarily selecting the whole window.
When the moving speed exceeds the upper limit value V even though the distance between the pointer and the window is within said predetermined range, and when the user stops moving the pointer (v=0), the operation picture selecting means determines that the user does not aim at this window and decides not to select the window by the prediction.
When the frame of the window is selected by the prediction, for example, the pointer picture modifying means changes the shape of the pointer from a unidirectional arrow head oriented to the direction of movement to a bidirectional arrow head which usually appears in changing the size of the window.
When the user effects clicking of a mouse button and the like while the pointer is in a state in which it is a bidirectional arrow head as described in the above, the operation input changes the size of the picture of the window.
Operation Picture Displaying Method
This invention provides an operation picture displaying method comprising the steps of; displaying one or more operation pictures each of which is correlated to an operation to a computer in a display screen, displaying a pointer at a pointed position in said display screen in response to an external operation pointing an arbitrary position in said display screen, detecting the direction and the speed of movement of said pointer in said display screen or the speed of movement of said pointer in said display screen, and the distance (pointer distance) between said pointer and each of said operation pictures displayed in said display screen, and selecting either one of said operation pictures displayed in said display screen based on said detected moving direction and moving speed.
First Recording Medium
This invention provides a first computer readable recording medium storing a program which causes a computer to execute: an operation picture displaying step of displaying one or more operation pictures each of which is correlated to an operation to a computer in a display screen, a pointer picture displaying step of displaying a pointer at a pointed position in said display screen in response to an external operation pointing an arbitrary position in said display screen, a moving direction/speed detecting step of detecting the direction and the speed of movement of said pointer in said display screen, and an operation picture selecting step of selecting either one of said operation pictures displayed in said display screen based on said detected moving direction and moving speed.
Second Recording Medium
This invention provides a second computer readable recording medium storing a program which causes a computer to execute: an operation picture displaying step of displaying one or more operation pictures each of which is correlated to an operation to a computer in a display screen, a pointer picture displaying step of displaying a pointer at a pointed position in said display screen in response to an external operation pointing an arbitrary position in said display screen, a moving speed/distance detecting step of detecting the speed of movement of said pointer in said display screen and the distance (pointer distance) between said pointer and each of said operation pictures displayed in said display screen, and an operation picture selecting step for selecting either one of said operation pictures displayed in said display screen based on said detected moving direction and moving speed.